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LONG-TERM  GOALS 

The  AP-UW  atmospheric  sciences  group  is  working  to  improve  forecaster  performance  at  Navy 
operational  weather  forecast  detachments  afloat  and  ashore.  This  work  encompasses  broad  research 
and  technology  development  in  areas  of  visualization,  human  factors,  human-machine  interaction,  and 
model  and  forecast  verification  with  an  emphasis  on  mesoscale  ensembles  and  visualization  of 
uncertainty. 

OBJECTIVES 

The  overall  objective  of  this  effort  is  to  develop  a  highly  automated,  rapid,  mesoscale  numerical 
weather  prediction  (NWP)  verification  tool  for  use  by  forecasters  and  model  developers.  The 
verification  technique  should  consider  distortion  errors  (phase/timing,  rotation,  and  stretching)  as  well 
as  the  normal  amplitude  errors.  It  is  intended  to  test  the  verification  tool  on  the  University  of 
Washington  Short  Range  Ensemble  Eorecast  System  (SREE)  and  with  a  version  of  the  Navy  COAMPS 
model  to  be  implemented  at  APE. 

APPROACH 

The  verification  technique  of  Van  Galen  (1970)  and  Hoffman,  et  ah,  (1995)  will  be  implemented  as  a 
rapid,  automated,  web-enabled  forecaster  and  model  developer  tool.  This  technique,  originally 
intended  for  synoptic-scale  features,  will  be  tested  on  mesoscale  predictions  of  a  number  of  parameters 
in  an  attempt  to  separate  model  forecast  errors  into  amplitude,  phase,  rotation,  and  distortion 
components  for  better  evaluation  of  mesoscale  model  capabilities.  In  order  to  improve  processing 
time,  image  motion  processing  techniques  (Chan,  1993;  Eim  and  Ho,  1998)  will  be  implemented  and 
tested  for  potential  acceleration  of  the  verification  routine.  The  verification  tool  will  be  used  to 
evaluate  the  University  of  Washington  Short  Range  Mesoscale  Ensemble  (SREE)  performance  and 
also  will  be  tested  on  an  APE  version  of  COAMPS  when  implemented.  As  an  expansion  of  this  effort, 
a  user-focused  cost-loss  evaluation  will  be  investigated  for  better  interpretation  of  component 
verification  (amplitude,  phase,  rotation,  distortion)  results. 
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WORK  COMPLETED 


During  the  first  year  of  the  projeet,  the  Hoffman  (1995)  teehnique  for  amplitude  and  phase  error  was 
implemented  in  both  MATLAB  and  web-enabled  versions  (see  figures  1  and  2  below)  and  fully  tested 
on  idealized  and  real  model  cases.  Three  image  motion-processing  techniques  were  implemented  in 
the  routine  and  tested  for  both  acceleration  and  accuracy.  Three  methods  of  error  minimization  (mean 
absolute  difference  (MAD),  route  mean  square  error  (RMSE)  and  mean  square  error  (MSE)  were 
tested  both  for  numerical  efficiency  and  accuracy  of  localization.  In  addition,  a  competing  verification 
technique  developed  by  University  of  Minnesota  was  tested  on  idealized  and  real  cases  and  a  paper  is 
currently  in  review. 

RESULTS 

The  Van  Galen/Hoffman  technique  has  been  successfully  implemented  both  in  MATLAB  for  further 
scientific  investigation  and  as  a  prototype  web-based  forecaster  tool  in  the  UW  Model  Uncertainty 
Monitor  (MUM).  The  MUM  is  a  prototype  forecaster  aide  being  developed  for  testing  at  NPMOE 
Whidbey  Island.  An  approximately  30x  increase  in  processing  speed  in  the  verification  tool  has  been 
achieved  by  implementing  a  combination  of  two  image  motion  processing  routines  and  optimizing  the 
computer  code  functions.  In  addition,  it  was  determined  that  MSE  error  minimization  for  localization 
produced  the  most  reliable  and  accurate  verification  computation  without  significantly  impacting 
computational  speed. 

IMPACT/APPLICATIONS 

Verification  systems  need  to  be  highly  automated  in  order  to  rapidly  assess  large  samples  of  cases; 
however,  they  must  also  be  able  to  correctly  evaluate  the  high  frequency,  high  amplitude  signals  of 
mesoscale  features.  Unfortunately,  traditional  methods  of  verification  have  been  shown  not  to  work 
for  mesoscale  numerical  weather  prediction.  Simple  automated  techniques  incorrectly  assess  slight 
phase  or  displacement  errors,  causing  smoothed,  or  ensemble  mean  forecasts  to  appear  to  perform 
better  than  a  more  detailed  deterministic  forecast,  yet  they  contain  less  forecast  content.  Case  studies, 
while  more  revealing,  are  too  time  consuming  to  assess  the  large  number  of  cases  required  for  subtle 
model  biases  or  differences  arising  from  small  changes  in  model  algorithms.  Mesoscale  NWP  forecast 
verification  is  a  critical  issue  for  US  Navy  operations.  More  NWP  model  outputs  are  becoming 
available  from  various  sources  and  it  is  difficult  for  management  and  operational  forecasters  to  choose 
the  appropriate  system  for  each  forecast  situation.  The  forecast  verification  tool  will  enable  more 
accurate  and  meaningful  evaluation  of  mesoscale  numerical  weather  prediction  systems,  especially 
mesoscale  ensemble  systems,  where  large  volumes  of  data  are  required  for  accurate  assessment  and 
where  small  prediction  distortions  or  displacements  cause  significant  misinterpretation  of  verification 
results.  The  tool  is  intended  for  use  both  by  model  developers  and  by  forecasters  for  quick  and  more 
accurate  model  assessment.  The  forecaster  tool  will  be  implemented  as  an  easy  to  use  web  tool. 


RELATED  PROJECTS 

The  University  of  Washington  Multidisciplinary  University  Research  Initiative  (MURI)  on  Integration 
and  Visualization  of  Multi-Source  Information  for  Mesoscale  Meteorology:  Statistical  and  Cognitive 
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Approaches  to  Visualizing  Uncertainty.  This  project  incorporates  a  number  of  verification  techniques 
into  a  forecaster  visualization  tool  and  a  prototype  version  of  our  mesoscale  verification  tool  has  been 
implemented  here. 
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Figure  1.  Mesoscale  Verification  Tool  Full  Map  Verification 
[Verification  of  the  UW  MM5  mesoscale  forecast  based  on  NOGAPS  initial  conditions  for 
November  F‘,  2002  with  mean  square  error  field  and  arrows  indicating  phase  error.] 
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Figure  2.  Mesoscale  Verification  Tool  with  User-Selected  Verification  Area 
[Verification  of  a  low-pressure  feature  demonstrating  user  selection  of  an  area  for  verification  and 
separation  of  mean  square  error  into  amplitude  and  phase  components.] 
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